Extract-and-Adaptation Network for 3D Interacting Hand Mesh Recovery

Understanding how two hands interact with each other is a key component of accurate 3D interacting hand mesh recovery. However, recent Transformer-based methods struggle to learn the interaction between two hands as they directly utilize two hand features as input tokens, which results in distant token problem. The distant token problem represents that input tokens are in heterogeneous spaces, leading Transformer to fail in capturing correlation between input tokens. Previous Transformer-based methods suffer from the problem especially when poses of two hands are very different as they project features from a backbone to separate left and right hand-dedicated features. We present EANet, extract-and-adaptation network, with EABlock, the main component of our network. Rather than directly utilizing two hand features as input tokens, our EABlock utilizes two complementary types of novel tokens, SimToken and JoinToken, as input tokens. Our two novel tokens are from a combination of separated two hand features; hence, it is much more robust to the distant token problem. Using the two type of tokens, our EABlock effectively extracts interaction feature and adapts it to each hand. The proposed EANet achieves the state-of-the-art performance on 3D interacting hands benchmarks. The codes are available at https://github.com/jkpark0825/EANet.Comment: Accepted at ICCVW 202

Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes

We consider the problem of recovering a single person's 3D human mesh from in-the-wild crowded scenes. While much progress has been in 3D human mesh estimation, existing methods struggle when test input has crowded scenes. The first reason for the failure is a domain gap between training and testing data. A motion capture dataset, which provides accurate 3D labels for training, lacks crowd data and impedes a network from learning crowded scene-robust image features of a target person. The second reason is a feature processing that spatially averages the feature map of a localized bounding box containing multiple people. Averaging the whole feature map makes a target person's feature indistinguishable from others. We present 3DCrowdNet that firstly explicitly targets in-the-wild crowded scenes and estimates a robust 3D human mesh by addressing the above issues. First, we leverage 2D human pose estimation that does not require a motion capture dataset with 3D labels for training and does not suffer from the domain gap. Second, we propose a joint-based regressor that distinguishes a target person's feature from others. Our joint-based regressor preserves the spatial activation of a target by sampling features from the target's joint locations and regresses human model parameters. As a result, 3DCrowdNet learns target-focused features and effectively excludes the irrelevant features of nearby persons. We conduct experiments on various benchmarks and prove the robustness of 3DCrowdNet to the in-the-wild crowded scenes both quantitatively and qualitatively. The code is available at https://github.com/hongsukchoi/3DCrowdNet_RELEASE.Comment: Accepted to CVPR 2022, 16 pages including the supplementary materia

Recovering 3D Hand Mesh Sequence from a Single Blurry Image: A New Dataset and Temporal Unfolding

Hands, one of the most dynamic parts of our body, suffer from blur due to their active movements. However, previous 3D hand mesh recovery methods have mainly focused on sharp hand images rather than considering blur due to the absence of datasets providing blurry hand images. We first present a novel dataset BlurHand, which contains blurry hand images with 3D groundtruths. The BlurHand is constructed by synthesizing motion blur from sequential sharp hand images, imitating realistic and natural motion blurs. In addition to the new dataset, we propose BlurHandNet, a baseline network for accurate 3D hand mesh recovery from a blurry hand image. Our BlurHandNet unfolds a blurry input image to a 3D hand mesh sequence to utilize temporal information in the blurry input image, while previous works output a static single hand mesh. We demonstrate the usefulness of BlurHand for the 3D hand mesh recovery from blurry images in our experiments. The proposed BlurHandNet produces much more robust results on blurry images while generalizing well to in-the-wild images. The training codes and BlurHand dataset are available at https://github.com/JaehaKim97/BlurHand_RELEASE.Comment: Accepted at CVPR 202

Dynamic Tilting of Ferroelectric Domain Walls via Optically Induced Electronic Screening

Optical excitation perturbs the balance of phenomena selecting the tilt orientation of domain walls within ferroelectric thin films. The high carrier density induced in a low-strain BaTiO3 thin film by an above-bandgap ultrafast optical pulse changes the tilt angle that 90{\deg} a/c domain walls form with respect to the substrate-film interface. The dynamics of the changes are apparent in time-resolved synchrotron x-ray scattering studies of the domain diffuse scattering. Tilting occurs at 298 K, a temperature at which the a/b and a/c domain phases coexist but is absent at 343 K in the better ordered single-phase a/c regime. Phase coexistence at 298 K leads to increased domain-wall charge density, and thus a larger screening effect than in the single-phase regime. The screening mechanism points to new directions for the manipulation of nanoscale ferroelectricity

Dynamic Tilting of Ferroelectric Domain Walls Caused by Optically Induced Electronic Screening

Optical excitation perturbs the balance of phenomena selecting the tilt orientation of domain walls within ferroelectric thin films. The high carrier density induced in a low-strain BaTiO3 thin film by an above-bandgap ultrafast optical pulse changes the tilt angle that 90{\deg} a/c domain walls form with respect to the substrate-film interface. The dynamics of the changes are apparent in time-resolved synchrotron x-ray scattering studies of the domain diffuse scattering. Tilting occurs at 298 K, a temperature at which the a/b and a/c domain phases coexist but is absent at 343 K in the better ordered single-phase a/c regime. Phase coexistence at 298 K leads to increased domain-wall charge density, and thus a larger screening effect than in the single-phase regime. The screening mechanism points to new directions for the manipulation of nanoscale ferroelectricity

Nanoscale Structural Characterization of Oxide and Semiconductor Heterostructures

According to a recent report from International Technology Roadmap for Semiconductors (ITRS), semiconductor industry based on silicon Complementary metal–oxide–semiconductor (CMOS) technology is facing challenges in terms of making the device faster with higher density and lower power consumption. To overcome the challenges, various methodologies are attempted using different state variables instead of electric charges, for example, polarization, phase states, and electron spin information. Different materials can also be chosen instead of silicon, for example, carbon, complex metal oxides in 1D or 2D nanostructure formations. A different concept of operating devices is also another option, for example, single electron transistors, spintronics, and quantum electronics. A tremendous number of stages during microfabrication manufacturing for integrated circuits consist of a series of deposition and etching processes. During these processes, unknown problems can arise from the design of their structural geometry. For example, unwanted strain distribution from the electrode patterns can change the electric properties of underlying materials regarding the decrease in charge carrier mobility or increase in leakage current in dielectrics, which all occur in nanoscale. So, it is important to understand the effects of structural phenomena on the electronic properties of materials using nanoscale characterization. The first work shows the changes in electronic property in Si quantum dot devices fabricated on Si/SiGe heterostructure is discussed. The electrode deposition process on the heterostructure surface is necessary for the device operation, but the electrodes also induce external nanoscale strain fields. These strain fields are transferred to the substrate materials via electrode edges and change electronic band structure. The magnitudes of the strain and their impact on changing the band structure are studied. In the second project, the alignment of ferroelectric polarization nanodomains in PbTiO3/SrTiO3 (PTO/STO) superlattice heterostructures is discussed. The PTO/STO nanostructure was created using a focused-ion beam technique. The domain alignment was observed using the x-ray nanodiffraction. A thermodynamic theoretical approach calculates the free energy density of the system to understand the origin of domain alignment. In the final project, the origin of photoinduced domain transformation in PTO/STO superlattices is discussed. Charged carriers are excited by the above-bandgap optical illumination, and transported by the internal electric fields arising from depolarization fields. These photoexcited charge carriers eventually screen the depolarization fields, and the initial striped nanodomain patterns transform to a uniform polarization state. After the end of illumination, the striped nanodomains patterns recover for a period of seconds at room temperature. The transformation time depends on the optical intensity, and the recovery time depends on the temperature. A charge trapping model with a theoretical calculation reveals that the charge trapping is a dominant process for the domain transformation, and the de-trapping process is for the recovery. Simulated domain intensity changes are in good agreements with the X-ray diffraction data.US Department of Energy Office of Basic Energy Scienc